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DETAILED ACTION 

Claim Rejections - 35 USC § 101 

1. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

Claims 1-20 and 22-25 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. The recited claims include the 
judicial exception of an abstract idea (determining a foreground color or determining 
legibility). No physical transformation is present to establish a practical application of 
the idea. The result (a foreground color or legibility measurement) is useful and 
concrete, but not tangible. 

Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

r 

3. Claims 1, 5-8 and 21-23 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Guo et al. (Hereinafter "Guo": Ping Guo and Michael R. Lyu, "A Study on Color Space 
Selection for Determining Image Segmentation and Region Number", Proceedings of 
the international Conference> IC-AI 2000, CSREA Press, Athens, GA, USA, Vol.3, pp. 
1127-1132, June 2000). 
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Regarding claims 1 and 22, Guo teaches a method and analogous system for 
automatically determining a foreground color for a digital image, comprising: (a) 
automatically dividing the colors of the pixels of at least a part of the digital image into 
a number of color clusters in a color space (see Abstract); and (b) for at least one 
color cluster, automatically selecting a color being related to the at least one color 
cluster according to predetermined criteria (§ 3, H 3). 

Regarding claim 5, Guo teaches the method according to claim 1, wherein said 
dividing at (a) comprises converting the image data to a predetermined color format (§ 
2.4, 1 2). 

Regarding claim 6, Guo teaches the method according to claim 1, wherein said 
dividing at (a) comprises using an Expectation-Maximization clustering (§ 2.1). 

Regarding claim 7, Guo teaches the method according to claim 1, wherein said 
dividing at (a) comprises determining the number of clusters using a model selection 
method one of a Bayesian Information Criterion and a Universal Model-based Minimum 
Description Length Principle (§ 2.3, i.e. Bayesian Probabilistic Classification). 

Regarding claims 8 and 23, Guo teaches the method and analogous system 
according to claim 1, further comprising automatically segmenting the part of the digital 
image into regions according to the color clusters before said selecting at (b) (§ 1, H 1 
and § 3, H 3). 
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Regarding claim 21, Guo teaches the method according to claim 1, further 
comprising one of displaying and storing a predetermined object using the selected 
color together with the digital image (§ 3 and Fig. 1). 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented 
and the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claim 2 is rejected under 35 U.S.C. 103(a) as being unpatentable over Guo as 
applied to claim 1 above, in view of Hamlin et al. (Hereinafter "Hamlin": US 5473738), 
and further in view of Sandow (US 5909220). 

Guo teaches the method according to claim 1. Guo does not teach the limitation 
wherein said selecting at (b) further comprises: selecting a harmonious color set with 
respect to the color clusters. 

However, Hamlin discloses a method wherein said selecting at (b) further 
comprises: selecting a harmonious color set with respect to the color clusters (Column 
1, lines 9-12). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine Hamlin's harmonious color selecting method with 
Guo's foreground color determining method, the motivation being to produce a pleasing 
effect to the eye (Hamlin: column 2, lines 25-27). 
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Guo in combination with Hamlin does not teach the limitation wherein said 
selecting at (b) further comprises testing the harmonious color set for legibility. 

However Sandow discloses a method wherein said selecting at (b) further 
comprises testing the harmonious color set for legibility (column 17, lines 25-29). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine Sandow's legibility testing method with the above 
combined teaching of Guo and Hamlin, the motivation being to control the legibility of 
lettering (Sandow: Column 3, lines 30-34). 

6. Claims 3 and 4 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Guo combined with Hamlin and in view of Sandow as applied to claim 2 above, and 
further in view of You et al. (Hereinafter "You": J. You, E. Pissaloux, H. A. Cohen, "A 
Hierarchical Image Matching Scheme Based on the Dynamic Detection of Interesting 
Points", International Conference on Speech and Image Processing, 1995, Vol. 4, pp. 
2467-2470). 

Regarding claim 3, Guo in combination with Hamlin and in view of Sandow 
teaches the method according to claim 2, as disclosed above, wherein said testing the 
harmonious color set for legibility further comprises: computing local measures of 
contrast between background and foreground in a neighborhood of a predetermined 
foreground region (Sandow: column 19, lines 1-8 and Fig. 12, background 65 in 
a neighborhood of foreground 64). 
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Guo in combination with Hamlin and in view of Sandow does not teach the 
limitation wherein said testing the harmonious color set for legibility further comprises 
computing a legibility score representative of a lowest few contrast values observed 
over the predetermined foreground region. 

However, You discloses computing a legibility score representative of a lowest 
few contrast values observed over the predetermined foreground region (page 2468, 
§ 2.1 and § 3, D 2, i.e. computing the Hausdorff distance (legibility score) 
using only the "interesting points" over the foreground region). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine You's computing method with the above combined 
teaching of Guo, Hamlin and Sandow, the motivation being to decrease computation 
time (You: § 3, H 1). 

Regarding claim 4, Guo in combination with Hamlin and in view of Sandow 
teaches the method according to claim 2, wherein a color is selected according to a 
legibility criterion for a predetermined foreground region by: computing local measures 
of contrast between background and foreground in a neighborhood for a predetermined 
foreground region (Sandow: column 19, lines 1-8 and Fig. 12, background 65 in 
a neighborhood of foreground 64). 

Guo in combination with Hamlin and in view of Sandow does not teach the 
limitation of computing a legibility score representative of a lowest few contrast values 
observed over the predetermined foreground region. 
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However You discloses computing a legibility score representative of a lowest 
few contrast values observed over the predetermined foreground region (page 2468, 
§ 2.1 and § 3, H 2, i.e. computing the Hausdorff distance (legibility score) 
using only the "interesting points" over the foreground region). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine You's computing method with the above combined 
teaching of Guo, Hamlin and Sandow, the motivation being to decrease computation 
time (You: § 3, H 1). 

7. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over Guo as 
applied to claim 8 above, in view of Shi et al. (Hereinafter "Shi": Jianbo Shi and Jitendra 
Malik, "Normalized Cuts and Image Segmentation", 2000, IEEE Transactions on Pattern 
Analysis and Machine Intelligence, Vol. 22, No. 8, pp. 888-905). 
Guo teaches the method according to claim 8. 

Guo does not teach the limitation wherein the segmenting comprises using one 
of a normalized cut criterion and an energy-minimization method. 

However, Shi discloses a segmentation method using one of a normalized cut 
criterion and an energy-minimization method (See Abstract). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine Shi's criterion with Guo's teaching, the motivation 
being to measure the goodness of an image segment (Shi: page 889, left column, 
lines 6-8). 
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8. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Guo as 
applied to claim 1 above, in view of Lovelady et al. (Hereinafter "Lovelady": US 
5335292). 

Guo teaches the method according to claim 1. 

Guo does not teach the limitation wherein said selecting at (b) further comprises: 
grouping the color clusters into interference clusters comprising a pixel in a selected 
region of the image and benign clusters comprising no pixel in the selected region; and 
selecting a color being related to all interference clusters according to predetermined 
criteria. 

However, Lovelady discloses a method wherein said selecting at (b) further 
comprises: grouping the color clusters into interference clusters comprising a pixel in a 
selected region of the image and benign clusters comprising no pixel in the selected 
region (column 13, lines 38-65); and selecting a color being related to all 
interference clusters according to predetermined criteria (column 13, lines 63-65, 
i.e. "keeper" color). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine Lovelady's grouping method with Guo's segmentation 
teaching, the motivation being to minimize the interference between colors (Lovelady: 
column 13, lines 30-32). 

9. Claim 11 is rejected under 35 U.S.C. 103(a) as being unpatentable over Guo as 
applied to claim 1 above, in view of Sandow. 
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Guo teaches the method according to claim 1. 

Guo does not teach the limitation wherein a color is selected according to a 
legibility criterion for a predetermined foreground region. 

However, Sandow discloses a method wherein a color is selected according to a 
legibility criterion for a predetermined foreground region (column 6, lines 55-60). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine Sandow's legibility testing method with the above 
teaching of Guo, the motivation being to control the legibility of lettering (Sandow: 
Column 3, lines 30-34). 

10. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over Guo 
combined with Sandow as applied to claim 11 above, in view of Gatica-Perez 
(Hereinafter "Gatica-Perez": Daniel Gatica-Perez, Zhi Zhou, Ming-Ting Sun and Vincent 
Hsu, "Video Object Hyper-Links for Streaming Applications", 2002, Springer-Verlag 
Berlin Heidelberg, VISUAL 2002, LNCS 2314, pp, 229-238). 

Guo in combination with Sandow teaches the method according to claim 11. 

Guo in combination with Sandow does not teach the limitation wherein a color is 
selected based on a likelihood ratio of the hypothesis that the digital image contains the 
foreground region and the hypothesis that the digital image does not contain the 
foreground region. 

However, Gatica-Perez discloses a method wherein a color is selected based on a 
likelihood ratio of the hypothesis that the digital image contains the foreground region 
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and the hypothesis that the digital image does not contain the foreground region (page 
232 and Fig. 2). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine Gatica-Perez's color selection method with the above 
combined teaching of Guo and Sandow, the motivation being to reduce the search 
space for segmentation (Gatica-Perez: § 2.3, H 1, lines 1-3). 
11. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over Guo 
combined with Sandow, in view of Gatica-Perez as applied to claim 12 above, and 
further in view of You. 

Guo in combination with Sandow and further in view of Gatica-Perez teaches the 
method according claim 12. 

Guo in combination with Sandow and further in view of Gatica-Perez does not 
teach the limitation wherein selecting a color comprises computing the legibility of 11 
min x T max y C 2 r(x+y), wherein 12 r ( x ) = h log Pr ( I ( x ) | T ) Pr ( I ( x ) | B ) 
,C.sub..epsilon..sup.2 is a disc of radius .epsilon and wherein Pr(I(x).vertline.T) denotes 
heuristic or other models of likelihoods that the image I contains text T at a given pixel 
x and Pr(I(x).vertline.B) denotes heuristic or other models of likelihoods that the image 
I contains background B at the given pixel x. 

However, You discloses a method wherein selecting a color comprises computing 
the legibility of 11 min x T max y C 2 r(x+y), wherein 12 r ( x ) = h log Pr ( I ( x ) | T ) 
Pr ( I ( x ) | B ) ,C.sub.. epsilon. .sup.2 is a disc of radius .epsilon and wherein 
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Pr(I(x).vertline.T) denotes heuristic or other models of likelihoods that the image I 
contains text T at a given pixel x and Pr(I(x).vertline.B) denotes heuristic or other 
models of likelihoods that the image I contains background B at the given pixel x (page 
2468, § 2.1 and § 3, H 2, i.e. computing the Hausdorff distance (legibility 
score) using only the "interesting points" over the foreground region, and 
using Gatica-Perez's likelihood ratio disclosure (Gatica-Perez: page 232 and 
Fig. 2)). 

12. Claims 14 and 15 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Guo as applied to claim 1 above, in view of Hamlin. 

Regarding claim 14, Guo teaches the method according to claim 1. Guo does 
not teach the limitation wherein a color is selected according to a color harmony 
criterion. 

However, Hamlin discloses a method wherein a color is selected according to a 
color harmony criterion (Column 1, lines 9-12). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine Hamlin's harmonious color selecting method with 
Guo's foreground color determining method, the motivation being to produce a pleasing 
effect to the eye (Hamlin: column 2, lines 25-27). 

Regarding claim 15, Guo in combination with Hamlin teaches the method 
according to claim 14, wherein a color is selected according to at least one of a 



Application/Control Number: 10/757,490 Page 12 

Art Unit: 2624 

monotonic, a complementary, and a p-adic color harmony criterion in HSL space 
(Hamlin: column 4, lines 25-27). 

13. Claims 16-18 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Guo combined with Hamlin as applied to claim 14 above, in view of Lovelady. 

Regarding claim 16, Guo combined with Hamlin teaches the method according 
to claim 14. 

Guo combined with Hamlin does not teach the limitation wherein a color is 
selected according to a color harmony criterion with respect to at least one interference 
cluster. 

However, Lovelady discloses a method wherein a color is selected according to a 
color harmony criterion with respect to at least one interference cluster (column 13, 
lines 63-65, i.e. "keeper" color). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine Lovelady's color selection method with the combined 
teaching of Guo and Hamlin, the motivation being to minimize the interference between 
colors (Lovelady: column 13, lines 30-32). 

Regarding claim 17, Guo combined with Hamlin teaches the method according 
to claim 14. 

Guo combined with Hamlin does not teach the limitation wherein a color is 
selected according to a color harmony criterion with respect to at least one benign 
cluster. 
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However, Lovelady discloses a method wherein a color is selected according to a 
color harmony criterion with respect to at least one benign cluster (column 13, lines 
38-65. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine Lovelady's color selection method with the combined 
teaching of Guo and Hamlin, the motivation being to minimize the interference between 
colors (Lovelady: column 13, lines 30-32). 

Regarding claim 18, Guo combined with Hamlin teaches the method according 
to claim 14. 

Guo combined with Hamlin does not teach the limitation wherein a color is 
selected according to a color harmony criterion with respect to at least one interference 
cluster and at least one benign cluster. 

However, Lovelady discloses a method wherein a color is selected according to a 
color harmony criterion with respect to at least one interference cluster (column 13, 
lines 63-65, i.e. "keeper" color) and at least one benign cluster (column 13, lines 
38-65. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine Lovelady's color selection method with the combined 
teaching of Guo and Hamlin, the motivation being to minimize the interference between 
colors (Lovelady: column 13, lines 30-32). 
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14. Claims 19 and 20 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Guo as applied to claim 1 above, in view of Hamlin, and further in view of Sandow. 

Regarding claim 19, Guo teaches the method according to claim 1. Guo does 
not teach the limitation wherein said selecting at (b) comprises determining a color 
subset according to a color harmony criterion and maximizing a legibility function in the 
color subset. 

However, Hamlin discloses a method wherein said selecting at (b) further 
comprises selecting a harmonious subset according to a color harmony criterion 
(Column 7, lines 19-22). 

. Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine Hamlin's harmonious color selecting method with 
Guo's foreground color determining method, the motivation being to produce a pleasing 
effect to the eye (Hamlin: column 2, lines 25-27). 

Guo in combination with Hamlin does not teach the limitation wherein said 
selecting at (b) further comprises maximizing a legibility function in the color subset. 

However Sandow discloses a method wherein said selecting at (b) further 
comprises maximizing a legibility function in the color subset (column 17, lines 25-29 
and column 36, lines 36-49, i.e. selecting color subset for maximum legibility 
slider value). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine Sandow's legibility maximization method with the 
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above combined teaching of Guo and Hamlin, the motivation being to control the 
legibility of lettering (Sandow: Column 3, lines 30-34). 

Regarding claim 20, Guo combined with Hamlin and Sandow teaches the 
method according to claim 1 and the harmonious color selection method (i.e. function) 
and legibility maximization function as recited in claim 19 above. 

Guo combined with Hamlin and Sandow does not disclose the limitation wherein 
a color c is selected for which 13i = lMll(c,Pi) + k= lNkh(c,Kk)is 
maximal, wherein P.sub.i denote the interference clusters, K.sub.k denote all clusters, 
both benign and interference, I is a legibility function in color space, h is a color 
harmony function, and .alpha. .sub.i and .gamma..sub.k are weighting factors. 

However, the examiner takes official notice that maximizing a weighted sum was 
notoriously well know in the art at the time of the invention. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to maximize a weighted sum comprising Guo's harmonious color 
selection function and Sandow's legibility maximization function, the motivation being to 
combine optimization of a pleasing effect to the eye (Hamlin: column 2, lines 25- 
27) and the legibility of lettering (Sandow: Column 3, lines 30-34). 
15. Claims 24 and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Sandow, and further in view of You. 

Regarding claim 24, Sandow teaches a method for determining legibility of an 
image having an identified foreground and background, comprising: computing local 
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measures of contrast between background and foreground in a neighborhood for a 
predetermined foreground region of the image (column 19, lines 1-8 and Fig. 12, 
background 65 in a neighborhood of foreground 64). 

Sandow does not teach the limitation of computing a legibility score 
representative of a lowest few contrast values observed over the predetermined 
foreground region. 

However, You discloses computing a legibility score representative of a lowest 
few contrast values observed over the predetermined foreground region (page 2468, 
§ 2.1 and § 3, % 2, i.e. computing the Hausdorff distance (legibility score) 
using only the "interesting points 7 ' over the foreground region). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to combine You's computing method with Sandow's teaching, the 
motivation being to decrease computation time (You: § 3, H 1). 

Regarding claim 25, Sandow in combination with You teaches the method 
according to claim 24, wherein the lowest few contrast values observed over the 
predetermined foreground region is a minimum of all local measures taken over the 
predetermined foreground region (page 2468, § 2.1 and § 3, H 2, i.e. computing 
the Hausdorff distance (legibility score) using only the "interesting points" 
over the foreground region). 
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Conclusion 

16. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Wu (US 2002/0027561) discloses color determination. 
Colmenarez et al. (Antonio J. Colmenarez and Thomas S. Huang, "Face Detection With 
Information-Based Maximum Discrimination", 1997, IEEE, pp. 782-787) discloses a 
visual learning application using a likelihood ratio. Sahoo et al. (P. K. Sahoo, S. Soltani, 
A. K. C. Wong and Y. C. Chen, "A Survey of Thresholding Techniques", 1988, Computer 
Vision, Graphics and Image Processing, Vol. 41, pp. 233-260) discloses local contrast 
techniques. Balasubramanian et al. (US 6414690) discloses contrast relationships in 
local areas. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Clifton G. Daley whose telephone number is 571-270- 
3144. The examiner can normally be reached on Monday - Friday 7:30am - 4:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Samir Ahmed can be reached on 571-272-7413. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO 
Customer Service Representative or access to the automated information system, call 
800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Samir Ahmed 
SPE 

Art Unit 2624 
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